Method for collision avoidance and laser machining tool

ABSTRACT

The invention relates to a method for collision avoidance of a laser machining head ( 102 ) in a machining space ( 106 ) of a laser machining tool ( 100 ), having the steps of; Monitoring a workpiece ( 112 ) in the machining space ( 106 ) with at least one optical sensor; Capturing images of the workpiece ( 112 ); Detecting a change in an image of the workpiece ( 112 ); Recognising whether the change comprises an object standing upright relative to the workpiece ( 112 ); Checking for a collision between the upright object and the laser machining head ( 102 ) based on a predetermined cutting plan and/or the current position (1016) of the laser machining head; Controlling the drives for moving the laser machining head ( 102 ) for collision avoidance in case of recognised risk of collision.

The invention relates to a method for collision avoidance of a lasermachining head and a numerically controlled laser machining tool. Inparticular, the invention relates to a method for collision avoidanceaccording to claim 1 and a numerically controlled laser machining toolaccording to claim 12.

A laser cutting machine separates flat sheets or pipes resting on acutting grid with a laser beam. Due to pressure surges of the cuttinggas or thermal stresses as well as unfavourable points of support on thegrate, cut parts can tip over and block the cutting head. It is alsopossible that the cut parts are completely released from the residualmaterial and flung through the interior of the machine. The followingapproaches to crash prevention in laser cutting machines are alreadybeing used.

Leaving micro-bridges: The outer contour of small parts is notcompletely cut. The parts are still attached at small bridges. At theend of the machining, the parts must be completely cut out or broken outin a further step.

The provision of an intelligent cutting contour: The cutting head is notmoved over already cut parts. This leads to more complex andlonger-lasting movement patterns.

The fragmentation of inner contours: The waste pieces are crushed inadvance so that they fall between the support points and cannot erect.This increases the energy consumption as well as the machining time.

The use of high-quality raw material: The use of high-quality rawmaterials can reduce the thermal expansion, but not the other sources oferror.

However, these approaches lead only to unsatisfactory results and do notachieve the desired reliability.

The object of the invention is to avoid the disadvantages of the priorart and to provide an improved laser machining tool. Alternative objectsare to provide an improved method for collision avoidance or an improvedlaser machining tool.

This object is achieved by a method according to claim 1 or anumerically controlled laser machining tool according to claim 12.

The method according to the invention for collision avoidance of a lasermachining head in a machining space of a laser machining tool comprisesthe steps:

monitoring a workpiece in the machining space with at least one opticalsensor;

Capturing images of the workpiece;

Detecting a change in an image of the workpiece;

Recognising whether the change comprises an object that stands uprightrelative to the workpiece;

Checking for a collision between the upright object and the lasermachining head based on a predetermined cutting plan and/or the currentposition of the laser machining head;

Controlling the drives for moving the laser machining head to avoidcollision in case of recognised risk of collision.

To avoid collisions between the cutting head and the workpieces whichlead to repairs and downtime of the machine, the collision avoidancemethod according to the invention proposes that the interior of themachine be monitored with at least one sensor. Suitable sensors are, forexample, cameras, such as ToF (time-of-flight) cameras or CCD/CMOScameras—preferably CMOS cameras. The live data from the sensors are usedto recognise an upright object, such as a cut part that has tilted orflown away. The combination of the planned track of the cutting head andthe upright object recorded by the sensors makes it possible torecognise collisions early. If such an event occurs, the machine willstop or, for example, bypass the critical point by raising the Z axis.For example, the one camera can comprise a dual sensor or stereo sensorto produce images from different angles or perspectives. These imagescan be recorded simultaneously or in quick succession, for example witha time interval of less than one second. Likewise, it is possible for acamera or lens to be spatially moved to produce images from differentangles or perspectives that provide information for depth representationor depth recognition. Multiple cameras can also be used. One or morechanges in an image of the workpiece relative to a chronologicallyearlier image of the workpiece are detected. The chronologically earlierimage can be the direct predecessor of the image. The image, in turn,later becomes the chronologically earlier image for a successor.

The collision avoidance method according to the invention has furtheradvantages in addition to the recognition of potential collisions andthe prevention of collisions. Thus, a visualisation of the cutting areafor the operator can result. In addition, the condition of the grid andtable and the workpiece position, the workpiece size, and damage can bedetermined.

It can be provided that measuring points are defined along a cuttingcontour of a cut part and monitored for brightness and/or colour values.This arrangement of measurement points enables the use of very efficientalgorithms such as the colour-along-edges algorithm. The number andexact arrangement of the measuring points can be adapted to thecircumstances, such as the geometry of the part to be cut, the cuttingspeed, the material thickness, or the like. All or some sides, such asonly one of two opposite sides, can be provided with measurement points.

It can further be provided that the images are captured offset in timeand that a change in an image of the workpiece is detected relative to achronologically earlier image of the workpiece. This enables fast imageprocessing.

It can be provided that a 3D object of the change is modelled and thatthe collision between the 3D object and the laser machining head ischecked. The live data from the sensors are used to continuously track a3D model of the cutting area. The combination of the planned track ofthe cutting head and the 3D topology recorded and calculated by thesensors makes it possible to recognise collisions early.

The method may include the further steps:

Calculating at least two shapes consisting of points and locatedparallel to a border of a cutting contour in an image, wherein one shapeis located inside the border and one shape is located outside theborder;

Extracting image pixels according to the points of the shapes;

Normalizing the image pixels by calculating a histogram of pixelbrightness for each shape;

Inputting the histograms into a deep neural network comprising an inputlayer, a plurality of internal layers and an output layer;

Processing the histograms with the deep neural network;

Outputting a variable by the deep neural network; and

Recognizing whether an object in the cutting contour is tilted for avalue of the variable being on a first side of a threshold or whether anobject in the cutting contour is not tilted for a value of the variablebeing on a second side of a threshold.

Such approach runs very quickly, thereby reducing reaction times. Thetwo shapes comprise or consist of a trace of points inside or outsidethe cutting contour. The offset to the cutting contour may be in therange of two to ten mm, preferably 5 mm. It is also possible to use morethan two shapes, e. g. two shapes inside and two shapes outside thecutting contour. Further, a reference image of the part or workpiecebefore cutting may be implemented as a further shape. A cutting plan maybe projected onto the image to define the cutting contours.

By calculating a histogram of pixel brightness the image pixels arenormalized. The brightness of a pixel is dependent from the reflectionof light at that pixel and is an indicator for the orientation or angleof the part on which the pixel is located. Reflection values may reachfrom dark/black (cut part absent or tilted) to bright/white (fullreflection). For example gray scale values from 0 to 256 may be binned.Each histogram may have between 12 and 64, preferably 32 bins, i.e.brightness values. The bins may have the same brightness range or havedifferent sizes or ranges for adapting the resolution or brightnessdistribution. Normalizing by a histogram results in the same amount ofpre-processed data regardless of the shape and the size of any contour.By reducing any number of pixels in a shape to a histogram with forexample only 32 values, any information about the size or shape of thecontour becomes irrelevant. Further, processing of such data with aNeural network is improved as all histograms have the same size, i.e.the numbers of bins.

Before inputting the histograms into a deep neural network this inputdata of histograms may be concatenated into a vector. The size of thevector may differs for the number of used cameras. For example, the sizeof the vector differs for two cameras and for one camera. For more thanone camera, the vector includes histograms and optionally 2D orco-occurrence histograms from each camera.

The neural network may be a deep neural network consisting of oneflattening layer as input layer, five internal dense layers with batchnormalization and one dense layer with sigmoid activation as an outputlayer. The deep neural network or models may differ for the number ofcameras only in the size of the input layer and in the number ofcoefficients. A model with at least two cameras, preferably two cameras,may be used. In cases where the cutting head is located between thecameras, the inference might be done with the camera that shows thecomplete contour.

The steps of outputting and recognizing may include:

Outputting, by the deep neural network, one floating point variable witha value in the range from 0,0 to 1,0 for the cutting contour; and

Recognizing whether an object in the cutting contour is tilted for avalue of the floating point variable being 0.5 or above or whether anobject in the cutting contour is not tilted for a value of the floatingpoint variable below 0.5.

It can be provided that two further shapes are calculated, wherein afirst further shape is located on the cutting contour and a secondfurther shape covers the whole area inside the cutting contour. Thepoints in the first further shape may be spaced very densely, like forexample at three points per mm. The points in the further three shapesmay be arranged sparser, like for example at two mm distance between twoadjacent points. The distance between points may be calculated in mm inthe image or in pixels of the image.

It can further be provided that before the step of extracting imagepixels, it is determined which image pixels are not covered by parts ofthe laser machining tool, for the determination a dynamic 3D model ofthe parts of the laser machining tool is provided and updated with livecoordinates from the laser machining tool, the visible, to be extracted,image pixels are calculated by comparison of the dynamic 3D model withthe images. For using AI based on vision, it may be determined whetherthe contour is visible or covered by parts of the machine. For thispurpose, a dynamic 3D model of the cutting machine consisting of bridge,support and cutting head may be implemented. The model may be updatedwith live coordinates, i.e. actual positions, velocities and/oraccelerations, from the machine and then used to calculate whichcontours or parts of contours are visible. It may further be calculatedwhich contours or parts of contours are visible from the utilizedcameras, e. g. from both cameras, only from the left camera, only fromthe right camera or not at all.

It can be provided that the step of normalizing the image pixels furtherincludes calculating a 2D histogram for the at least two shapes. Such 2Dor co-occurrence histogram is calculated for shapes inside and outsideof the cutting contour. This 2D histogram uses the multiplication of thebins for the histograms, e.g. 32 by 32 bins. The 2D histogram puts thebrightness of corresponding points inside and outside of the cut incorrelation to improve the processing along the cut lines.

It can further be provided that the recognition is based on alreadypre-calculated possible positions of cut parts of the workpiece. Forthis purpose, the contours of the cut parts, that is the parts to be cutout, are rotated and stored in possible positions. The rotation can takeplace about an axis, for example the vertical axis or z-axis, or alsoabout several axes.

It can be provided that a cut part is identified and the positionthereof compared with already calculated possible positions of this cutpart. Then only a simple matching or comparison algorithm is requiredduring the runtime or machining time, which saves time and thus makesthe process faster.

It can also be provided that the laser machining head is driven tobypass the change or to stop. If no further bypassing is possible due tothe speed, acceleration, and/or position of the laser machining head orthe position of the change, that is an upright part of the workpiece ora separated part, a stop or emergency stop can be controlled to avoid acollision. If a collision is avoidable, the change is detoured around orbypassed. Then the laser machining head is driven accordingly.

The numerically controlled laser machining tool according to theinvention with a machining space for receiving metallic workpieces to bemachined and a laser machining head for machining the workpiecescomprises

a numerical control unit,

an optical sensor system having at least one optical sensor whichcaptures at least a part of the machining space and the workpiecearranged therein,

a graphics processing unit connected to the optical sensor system andconfigured to process data from the sensor to recognise changes in theworkpiece, and connected to the numerical control unit, wherein thegraphics processing unit is configured to recognise whether a changecomprises an upright object, wherein the graphics processing unit and/orthe numerical control unit is configured to check for collision betweenthe upright object and the laser machining head based on a predeterminedcutting plan and/or the current position and/or the trajectory of thelaser machining head, and

-   -   wherein the numerical control unit is configured for collision        avoidance in the event of a recognised risk of collision.

The graphics processing unit is preferably configured for real-timeimage processing and ideally comprises one or more CPUs and/or GPUs.Particularly suitable are highly parallel GPUs with 256 or more cores.Otherwise the same advantages and modifications apply as describedabove.

It can be provided that the optical sensor system comprises two or atleast four cameras, preferably CMOS cameras. The use of CMOS cameras orimage acquisition units without image processing enables a very highprocessing speed, so that sufficient reaction time is available even athigh speeds of the laser machining head. With four or more cameras, forexample, a better resolution can be achieved and parallax errors can bereduced.

It can further be provided that two cameras are provided, the captureareas of which are aligned in the same direction, and that the capturearea of a first camera captures a first half of the machining space andthat the capture area of a second camera captures a second half of themachining space. With this arrangement, the shadowing by the lasermachining head is reduced since at least one camera always has a freefield of view.

It can be provided that four cameras are provided and that the captureareas of two cameras capture a first half of the machining space andthat the capture areas of two cameras capture a second half of themachining space, wherein the two cameras are each offset from eachother. The combination of both perspectives or both capture areas makesit possible to obtain information about the depth of the observedobject. Ideally, cameras are installed on both sides of the cutting areato refine the evaluation.

It can be provided that the camera or the cameras are connected to thegraphics processing unit with a high-speed connection. To respond to acritical situation with minimal delay, the high-speed link provideslow-latency transmission. In addition, the high-speed connection canbridge a few meters between the cameras and the graphics processingunit. The high-speed connection can comprise, for example, opticalfibre, coaxial cable, twisted pair, etc. A new bus for video data is,for example, the FPD link, which is used in vehicles to drive displays.Such links or connections allow a high data transfer rate, for examplegreater than 1 GHz, for the transmission of many high-resolution images.

It can be provided that the graphics processing unit is connected to thenumerical control unit with a real-time Ethernet connection. A real-timeEthernet connection, such as EtherCAT (Ethernet for Control AutomationTechnology), enables real-time image data availability for real-timeprocessing by the graphics processing unit.

It can further be provided that the camera system and/or the graphicsprocessing unit is configured to perform a first calibration of theintrinsic camera parameters and a second calibration of translation androtation parameters of a coordinate system of the camera compared to acoordinate system of the laser machining tool. The calibration of theintrinsic camera parameters is complex and not automated. A single passcan be sufficient as long as the lens on the camera is not adjusted. Forthis calibration, images of for example a chessboard at different anglesare needed. The intrinsic parameters are then calibrated with imageprocessing and these images. This calibration creates the software modelof the camera and lens. The calibration of the translation and rotationparameters can be repeated with each movement of the camera or thefixtures thereof. This calibration is easy to automate, so it isrecommended to periodically recalibrate these parameters. Movements overtime are to be expected due to of vibrations or slight thermaldeformation of the machine housing. At least four points in the machinecoordinate system and on the image must be known for this calibration.For this calibration, a Harris corner of sufficient size can be attachedto the cutting head. This Harris corner can be recognised with thecameras and compared with the current cutter head coordinate.Corresponding machine and image coordinates can be determined.

Further preferred embodiments of the invention will become apparent fromthe remaining features mentioned in the dependent claims.

The various embodiments of the invention mentioned in this applicationare, unless otherwise stated in the individual case, advantageouslycombinable with each other.

The invention will be explained below in exemplary embodiments withreference to the accompanying drawings. In the figures:

FIG. 1 shows a schematic perspective view of a numerically controlledlaser machining tool;

FIG. 2 shows a schematic representation of a control of the numericallycontrolled laser machining tool of FIG. 1;

FIG. 3 shows a schematic representation of two cameras of the lasermachining tool for capture of the machining space;

FIG. 4 shows a schematic representation of two other cameras of thelaser machining tool for capture of the machining space;

FIG. 5 shows a schematic representation of the capture areas of the fourcameras of FIG. 4;

FIG. 6 shows a schematic representation of a flown-away part of aworkpiece;

FIG. 7 shows a schematic representation of a cut-out part of a workpiecewith measuring points;

FIG. 8 shows a schematic representation of the cut-out part of FIG. 7showing the part extracted by image processing;

FIG. 9 shows a schematic representation of the matching of the extractedpart;

FIG. 10 shows a flowchart of a method for collision avoidance of a lasermachining head;

FIG. 11 shows a flow chart of a general method for collision avoidanceof a laser machining head; and

FIG. 12 shows an exemplary depiction of shapes of a cutting contour.

FIG. 1 shows a schematic perspective view of a numerically controlledlaser machining tool 100, in particular a laser cutting machine with alaser machining head 102, in particular a laser cutting head. The lasercutting head 102 is arranged on a movable bridge 104 so that it can bemoved in at least the x and y directions in a machining space 106 of thelaser machining tool 100. A laser source 108 generates laser light andsupplies it to the laser cutting head 102 via a light guide 110. Aworkpiece 112, for example a metal sheet, is arranged in the machiningspace 106 and is cut by the laser beam.

FIG. 2 shows a schematic representation of a controller 200 of thenumerically controlled laser machining tool 100 from FIG. 1. A numericalcontrol unit 202, also called CNC (Computerised Numerical Control),executes the cutting plan as an EtherCAT master 204 in that the positionsignals are output via an EtherCAT bus 206 to the drives 208 as EtherCATslave 210. One of the drives 208 is exemplified as EtherCAT slave 210.This EtherCAT slave 210 and other EtherCAT slaves write data, forexample from sensors, such as incremental encoders, to the EtherCAT bus206, and read data, which for example is used to control outputs, fromthe EtherCAT 206 bus.

In this example, four cameras 212 are provided, the arrangement of whichin the machining space 106 of the numerically controlled laser machiningtool will be explained in more detail in the following figures.Preferably, CMOS cameras or image recording units are provided withoutimage processing, which enables a very high processing speed.

The image data of the cameras 212 are forwarded to a graphics processingunit 214 where the processing of the image data takes place. Thegraphics processing unit 214 preferably comprises a plurality, forexample, 512 or more GPUs, and is preferably configured for real-timeimage processing. Particularly suitable are highly parallel GPUs with256 or more cores. The graphics processing unit 214 also operates asEtherCAT slave 210 and thus is in direct communication with numericalcontrol unit 202.

The graphics processing unit 214 and/or the numerical control unit 202are configured to carry out the methods or operations illustrated inFIGS. 6 through 10 and described below. In particular, the graphicsprocessing unit 214 is configured to process data from the cameras 212to recognise changes to the workpiece, to model a change from a 3Dobject, and, optionally together with the numerical control unit 202, tocheck for collision between the 3D object and the laser machining headbased on a predetermined one cutting plan and/or the current position ofthe laser machining head. In addition, the numerical control unit 202 isconfigured for collision avoidance in the event of a recognised risk ofcollision.

The graphics processing unit 214 obtains the cutting geometry ortrajectory of the laser cutting head from the numerical control unit 202via the EtherCAT bus 206. Before a collision event occurs, the graphicsprocessing unit 214 can signal this via the EtherCAT bus 206. Thesignalling can be sent to the numerical control unit 202 and/or directlyto the drive(s) 208 for the fastest possible response, such as emergencystop or bypass.

This can be done gradually depending on the time available up to acollision. If there is sufficient time for an evasive manoeuvre, thegraphics processing unit 214 sends data, such as the position orcoordinates of the collision to the numerical control unit 202, which inturn calculates an evasive route and drives the drives 208 accordingly.The new alternate route is also sent to the graphics processing unit214, which now continues to check the new route for collision.

If there is insufficient time for an evasive manoeuvre, the graphicsprocessing unit 214 sends emergency stop commands directly to the drives208 to achieve the fastest possible stop of the laser cutting head.

A computing unit 216 of the graphics processing unit 214 can be realisedeither by means of a CPU, a graphics processing unit GPU, or acombination of both. The computing unit 216 has enough computing powerto evaluate the received camera data in real time and to make a decisionas to whether a collision is imminent. This must be done fast enoughthat the numerical control unit 202 of the machine can take appropriateaction to avoid the collision. The computing unit 216 or the graphicsprocessing unit 214 is connected to the numerical control unit 202, forexample via the illustrated EtherCAT bus 206.

All elements of the controller 200, in particular the graphicsprocessing unit 214, the cameras 212, and the bus 206, are configuredfor a real-time capability of the system.

FIGS. 3 through 5 show schematic representations of a camera system 300of the numerically controlled laser machining tool 100 with at least twocameras 212. In addition to the cameras 212, suitable illuminations, forexample LED lights, can be provided to enhance the quality of the cameraimages.

FIG. 3 shows two cameras 212 for which the capture areas 302 are alignedin the same direction. The capture area 302 of a first camera 212captures a first half of the workpiece 112 or of the machining space106. The capture area 302 of a second camera 212 captures a second halfof the workpiece 112 or of the machining space 106. Thus, the twocameras capture the entire machining space 106. The two cameras 212 arearranged laterally offset from a longitudinal axis A of the machiningspace 106, so that the capture areas 302 extend laterally or obliquelyinto the machining space 106.

FIG. 4 shows a further schematic representation of the camera system 400of the numerically controlled laser machining tool 100. Here, the twocameras 212 are arranged in a mirrored manner in comparison to thearrangement of FIG. 3 on the longitudinal axis A of the machining space106. Likewise, the capture areas 402 are inclined and, in comparison toFIG. 3, aligned opposite to the longitudinal axis A. Analogous to FIG.3, the capture area 402 of a first camera 212 captures a first half ofthe workpiece 112 or of the machining space. The capture area 402 of asecond camera 212 captures a second half of the workpiece 112 or of themachining space.

FIG. 5 shows a further schematic illustration of the camera system 500of the numerically controlled laser machining tool 100 with the captureareas 302 and 402 of the four cameras (not shown here).

In this example, cameras are installed on both sides of the cutting areaor machining area to refine the evaluation. The combination of bothviewpoints 302 and 402 provides information about the depth of theobserved object. This depth or spatial information enables the modellingof a 3D object from a change in the workpiece 112.

FIG. 6 shows a schematic illustration of a flown-away part or cut part600 of a workpiece. The cut part 600 shown here is located next to thecutting contour 602, where the cut part 600 was originally located, thatis before the cutting. The illustration shown here can be, for example,the shot from a single camera.

Such cut parts 600, which fly away due to the gas pressure and landanywhere on the raw material or the workpiece 112, can be detected inthat first a reference depiction of the workpiece 112 is created andthen current depictions or shots are continuously compared with thereference depiction. This can be done in particular at the points wherethe raw material has not yet been processed. If a position or change inthe comparisons is classified as critical, the exact position, inparticular the height, of the part resting over the workpiece 112 can bedetermined with a 3D fitting.

As a possible remedy in a critical classification, that is, a potentialcollision between the cut part 600 and the laser machining head, theflown-away cut part 600 can be blown away with gas pressure, this areawill not be cut, or the operation will be discontinued.

FIG. 7 shows a schematic representation of a cut-out part or cut part700 of a workpiece 112. The two illustrations at the top of FIG. 7 canin turn be shots from one or more cameras. The lowermost illustration inFIG. 7 depicts a colour-along-edges algorithm for detecting a change inan image of the workpiece 112.

For the colour-along-edges algorithm, a very accurate projection of 3Dpoints in the machine coordinate system onto the 2D images is desirable.For this, the cameras 212 must be calibrated. Image processing executedin, for example, the graphics processing unit 214 is used for thecalibration and projection. Two different calibrations are performed.The first is the calibration of the intrinsic camera parameters. Thesecond calibration is the calibration of the translation and rotationparameters in the coordinate system of the camera 212 compared to thecoordinate system of the machine 100.

The calibration of the intrinsic camera parameters is complex and notautomated. A single pass can be sufficient as long as the lens on thecamera 212 is not adjusted. For this calibration, images of a chessboardat different angles are needed. The intrinsic parameters are thencalibrated with image processing and these images. This calibrationcreates the software model of the camera and lens.

The calibration of the translation and rotation parameters can berepeated with each movement of the cameras 212 or the fixtures thereof.This calibration is easy to automate, so it is recommended toperiodically recalibrate these parameters. Movements over time are to beexpected due to of vibrations or slight thermal deformation of themachine housing. At least 4 points in the machine coordinate system andin the image must be known for this calibration.

A Harris corner of sufficient size can be attached to the cutting headas a target for this calibration. This Harris corner can be recognisedwith the cameras 212 and compared with the current cutter headcoordinate. Corresponding machine and image coordinates can beconnected.

The target, for example a Harris corner, is preferably attached to thecutting head. This target can be recognised automatically if itsapproximate position on the image is known. This is the case with aperiodic recalibration.

For the calibration process, therefore, the following steps areperformed respectively. First, the cutting head is positioned in fourdefined positions. At each of these positions, one mage is taken witheach of the two cameras or two viewing angles. On each image, the imagecoordinates of the Harris corner are determined. From the machinecoordinates of the four positions and the image coordinates of theHarris corner, the translation and rotation parameters are calculated.

From the workpiece 112, the cut part 700 is cut out by means of a laserbeam 702. This process is observed by the cameras. Measurements aretaken at certain measuring points 704 along the sectional contour in animage processing executed in the graphics processing unit 214. Themeasurement points 704 are used to detect a change in an image of theworkpiece 112.

Now, when the cut part 700 tilts, the changes in the amount of lightalong the contour 706 reaching the camera are detected in the firststep. This change in the amount of light occurs through the change inthe reflection angle of the cut part 700. This can mean both additionalbrightness and reduced brightness.

The tilted cut part 700 partially disappears under the remainingworkpiece 112, resulting in a strong contrast. Here the contrast isshown as a change between white and black. In fact, changes in colourvalues, brightness values and/or contrast values can be used.

These changes are analysed and a check is made to see if a threshold tostart further processing has been reached. According to the illustrationat the bottom of FIG. 7, the difference between the colour value of ameasuring point 704 lying within the contour 706 and a point locatedoutside the contour 706 is determined and then evaluated.

If both colour values of the reflected light are the same or have only avery slight deviation, then the cut part 700 is not tilted (FIG. 7, top)or the tilted cut part 700 and the remaining work piece 112 are at aboutthe same height (FIG. 7, middle), such as in the lower left corner ofthe cut part 700. In this case, there is no risk and the value is belowthe threshold. Further action is not necessary and monitoring will becontinued.

If both colour values are different, then the cut part 700 at thesemeasuring points is no longer within the contour 706 (FIG. 7, centre),for example, in the upper left corner of the cut part 700. In this case,there is also no risk and the value is below the threshold. Furtheraction is not necessary and monitoring will be continued.

If both colour values are partly different, then the cut part 700 islocated at these measurement points outside the contour 706 and abovethe remaining workpiece 112 (FIG. 7, centre), such as in the upper rightcorner of the cut part 700. In this case, there is a risk of a collisionsince the cut part 700 rises and the threshold is exceeded.

The threshold for starting the second algorithm is then reached. Thesecond algorithm is called 3D fitting and will be described withreference to FIGS. 8 and 9. In contrast to the colour-along-edgesalgorithm, in which a change and thus potential risk are quicklyrecognised, the 3D fitting involves the recognition of whether thecorresponding part actually poses a risk to the cutting head and thusfor the machining process. It is quite possible that a change isdetected, but it does not turn out to be a risk. Such cases do not leadto a stopping of the machining process due to this bifurcation of thealgorithm.

FIG. 8 shows a schematic representation of the cut-out part of FIG. 7,showing the part extracted by image processing. The contour 800 of theupright cut part 700 in the camera image is determined. For thispurpose, subtraction algorithms are used, for example. Thisdetermination also takes place in the graphics processing unit 214.

FIG. 9 shows a schematic representation of the matching of the extractedpart. The matching or comparison also takes place in the graphicsprocessing unit 214.

From the cutting plan, first the critical cut part 700 which wasdetected in the colour-along-edge algorithm is selected. In contrast tothe camera image (see FIG. 8), the complete contour 900 of the part 700is obtained from the cutting plan.

A possible matching algorithm works as described below.

The original contour 900 is rotated by the 3D fitting algorithm alongone, several, or all three axes. Thus, the contour 900 is modelled inall possible positions in which the cut part 700 can lie. By way ofexample, the contours 900 a, 900 b, and 900 c are shown here.

This modelling of the contours 900 a, 900 b, and 900 c or the cuttingparts can be done before the start of cutting, so that the algorithm isas efficient as possible during testing, since what must be done is onlythe comparison but not the modelling.

Now, when the information of the model is available and a cut parttilts, the contour 800 of the tilted part 700 recognised by the camerais compared with the models 900 a, 900 b, and 900 c.

The best match between the model and the contour 800 of the tilted part700 is defined. Here it is the contour 900 a. From this, it can then becalculated at which position and by how much the part stands upright.Together with the information on where the cutting head will move withinthe next few seconds, it can be calculated whether or not a collision ispossible.

If a collision is possible, the area around the position is marked as arisk zone. Now it must be decided what the control unit should initiateas a countermeasure. The collision can be prevented, for example, bystopping the machine rapidly. The even more efficient solution is thatthe cutting head either drives around the risk zone, lifts up to avoidthe collision, or a combination of both.

FIG. 10 shows a flow chart of a method for collision avoidance by alaser machining head in a machining space of a laser machining tool.

In a first step 1000, camera data are generated, i.e., images of theworkpiece with at least one optical sensor, preferably two, four, ormore sensors.

In a second step 1002, changes are detected in an image of the workpieceby means of a previously described colour-along-edges algorithm. If alocal change is detected in step 1004, the method proceeds to block1010. If not, then branching back to the monitoring in step 1002 resultsin a monitoring loop. This algorithm detects local changes in cut partsor the like, and not global changes, such as feeding or removing aworkpiece. The two steps 1002 and 1004 are part of a local changerecognition process.

The numerical control process 1012 is executed in the numerical controlunit. The numerical control unit knows the cutting plan 1014 and thecurrent position 1016 of the cutting head, and in step 1018 calculatesthe planned track or route of the cutting head from the given cuttingplan 1014 and/or the current position of the laser machining head.

The cutting plan 1014 is supplied to a process for modelling theinterior space or the machining space. This process, as well as thelocal change recognition process, operates in the collision monitoringsystem formed in or executed by the graphics processing unit.

The block 1006 of the interior modelling process is supplied with thecutting plan 1014. A topology of the interior or the workpiece to bemachined is created from the cutting plan 1014. The topology comprisesthe workpiece as well as the cutting pattern planned on the workpieceand can comprise the respective circumferences and locations of the cutparts. This topology is supplied to block 1010.

In block 1010, the 3D fitting is carried out, that is to say themodelling of a 3D object of the change, as described above. For thispurpose, the camera data 1000 is supplied to the block 1010. The 3Dfitting is started when a local change is detected in block 1004. As theoutput of the modelling, a 3D topology 1008 of the change is provided,such as a contour 800.

This 3D topology 1008, like the planned track 1018, is supplied to aprocess collision detector. This process is formed in or executed by thegraphics processing unit.

The 3D topology 1008 and the planned track 1018 are supplied to acollision detector 1020, an algorithm in the graphics processing unit,and/or the numerical control unit 1012. The collision detector 1020checks to see if the 3D topology 1008 is within the planned track. If itis determined in step 1022 that a collision is possible, the methodproceeds to block 1024. If not, then branching back to the monitoring instep 1002 (not shown) results in a monitoring loop. The block or step1002 is executed continuously.

In step 1024, a countermeasure is taken by driving the laser machininghead for collision avoidance in the event of a recognised risk ofcollision. The countermeasure is a stop and/or evasion of or bypassingthe obstacle. Blocks 1020, 1022, and 1024 are part of the collisiondetector process.

The result of step 1024 is supplied to the CNC process 1012 forprocessing and implementation. For example, for an emergency stop, thedrives of the laser machining head can also be controlled directly, thatis without the involvement of the numerical control unit.

FIG. 11 shows a flow chart of a general method for collision avoidanceof a laser machining head. In a first step 1100, the entire cutting areaor the entire machining space is continuously monitored by the sensorsystem.

Checks for a local change as stated above are also made continuously instep 1101. A local change is usually caused by a cutting process. Ifthere is no local change, branching back to step 1100 results in amonitoring loop.

If a local change is recognised, the method proceeds to step 1102 wherethe change is analysed as outlined above.

FIG. 12 shows an exemplary depiction of shapes of a cutting contour likethe cutting contour 602 of FIG. 6. For the cutting contour, the systemcalculates four sets of points, called shapes 1200, 1202, 1024, and1206. The shapes 1200, 1202, 1204, and 1206 may be arranged on an imageof the workpiece or the cutting contour.

A first shape 1200 consists of points lying on the actual cutting line.A second shape 1202 consists of a trace of points inside the cuttingcontour, at an offset of five mm. A third shape 1204 consists of a traceof points outside the cutting contour, also at an offset of for examplefive mm. A fourth shape 1206 covers the whole area inside of the cuttingcontour.

Image pixels of the four shapes 1200, 1202, 1204, and 1206 are extractedfrom the image. In a normalization step, histograms of pixel brightnessare calculated for each of the four shapes 1200, 1202, 1024, and 1206.Each histogram has for example 32 bins. In addition, a co-occurancehistogram is calculated for the second shape 1202 and the third shape1204 three. This co-occurance or 2D histogram includes 32 by 32 bins andputs the brightness of corresponding points inside and outside of thecut in correlation. The x axis of the 2D histogram may be the secondshape 1202 and the y axis of the 2D histogram may be the third shape1204.

Then, concatenation of this input data into a vector is calculated. Thesize of the vector differs for the Neural network for two cameras andthe Neural network for one camera.

The Neural network accepts input data as a vector containing theconcatenated histograms. For the Neural network that predicts contoursvisible by two cameras, the following sequence is used:

-   -   Shape 1200 histogram from right camera (32 values)    -   Shape 1200 histogram from left camera (32 values)    -   Shape 1202 histogram from right camera (32 values)    -   Shape 1202 histogram from left camera (32 values)    -   Shape 1204 histogram from right camera (32 values)    -   Shape 1204 histogram from left camera (32 values)    -   Shape 1206 histogram from right camera (32 values)    -   Shape 1206 histogram from left camera (32 values)    -   Co-occurance histogram from right camera (32×32=1024 values)    -   Co-occurance histogram from left camera (32×32=1024 values)

This totals up to 2304 input values for the Neural network.

For the Neural network that predict contours visible only by one camera,the sequence is as follows:

-   -   Shape 1200 histogram (32 values)    -   Shape 1202 histogram (32 values)    -   Shape 1204 histogram (32 values)    -   Shape 1206 histogram (32 values)    -   Co-occurance histogram (32×32=1024 values) This totals up to        1152 input values for the Neural network.

The Neural network is in this example a deep Neural network consistingof one flattening layer as input layer, five internal dense layers withbatch normalization and one dense layer with sigmoid activation asoutput layer.

From the normalized and concatenated input data, the deep Neural networkoutputs one floating point value in the range from 0.0 to 1.0 percontour. If the value is below 0.5, the contour is predicted to be safe.If the value is 0.5 or above, the contour is predicted to be dangerouslytilted.

In FIG. 10 showing a flowchart of a method for collision avoidance of alaser machining head, the above described implementation of the Neuralnetwork may replace the interior modelling (steps 1006 and 1010) and thestep 1008. Alternatively, the above described implementation of theNeural network may replace the recognition of local changes (steps 1002and 1004), the interior modelling (steps 1006 and 1010), and the step1008.

The method presented here for collision avoidance by a laser machininghead in a machining space of a laser machining tool enables a simple andprecise recognition of possible obstacles in the planned track in realtime and a collision avoidance in case of recognised risk of collision.

1. A method for collision avoidance of a laser machining head in amachining space of a laser machining tool, having the steps of:Monitoring a workpiece in the machining space with at least one opticalsensor; Capturing images of the workpiece; Detecting a change in animage of the workpiece; characterized by Recognising whether the changecomprises an object standing upright relative to the workpiece; Checkingfor a collision between the upright object and the laser machining headbased on a predetermined cutting plan and/or the current position of thelaser machining head; Controlling the drives for moving the lasermachining head for collision avoidance in case of recognised risk ofcollision.
 2. The method according to claim 1, characterised in thatmeasuring points are defined along a cutting contour of a cut part andmonitored for brightness and/or colour values.
 3. The method accordingto claim 1, characterised in that the images are captured offset in timeand that a change in an image of the workpiece is detected relative to achronologically earlier image of the workpiece.
 4. The method accordingto claim 1, characterised in that a 3D object of the change is modelledand is checked for collision between the 3D object and the lasermachining head.
 5. The method according to claim 1, characterised by thefurther steps: Calculating at least two shapes consisting of points andlocated parallel to a border of a cutting contour in an image, whereinone shape is located inside the border and one shape is located outsidethe border; Extracting image pixels according to the points of theshapes; Normalizing the image pixels by calculating a histogram of pixelbrightness for each shape; Inputting the histograms into a deep neuralnetwork comprising an input layer, a plurality of internal layers and anoutput layer; Processing the histograms with the deep neural network;Outputting a variable by the deep neural network; and Recognizingwhether an object in the cutting contour is tilted for a value of thevariable being on a first side of a threshold or whether an object inthe cutting contour is not tilted for a value of the variable being on asecond side of a threshold.
 6. The method according to claim 5,characterised in that two further shapes are calculated, wherein a firstfurther shape is located on the cutting contour and a second furthershape covers the whole area inside the cutting contour.
 7. The methodaccording to claim 5, characterised in that before the step ofextracting image pixels, it is determined which image pixels are notcovered by parts of the laser machining tool, for the determination adynamic 3D model of the parts of the laser machining tool is providedand updated with live coordinates from the laser machining tool, thevisible, to be extracted, image pixels are calculated by comparison ofthe dynamic 3D model with the images.
 8. The method according to claim5, characterised in that the step of normalizing the image pixelsfurther includes calculating a 2D histogram for the at least two shapes.9. The method according to claim 1, characterised in that therecognition is based on already pre-calculated possible positions of cutparts of the workpiece.
 10. The method according claim 1, characterisedin that a cut part is identified and compared with its position withalready calculated possible positions of this cut part.
 11. The methodaccording claim 1, characterised in that the laser machining head isanticipated by means of the trajectory thereof to bypass the change orto stop.
 12. A numerically controlled laser machining tool having amachining space for receiving metallic workpieces to be machined and alaser machining head for machining the workpieces, comprising anumerical control unit, an optical sensor system having at least oneoptical sensor which captures at least a part of the machining space andthe workpiece arranged therein, a graphics processing unit connected tothe optical sensor system and configured to process data from the sensorto recognise changes to the workpiece, and which is connected to thenumerical control unit, characterized in that the graphics processingunit is configured to recognise whether a change comprises an objectstanding upright relative to the workpiece, that the graphics processingunit and/or the numerical control unit are configured to check forcollision between the upright object and the laser machining head basedon a predetermined cutting plan and/or the current position and/or thetrajectory of the laser machining head, and that the numerical controlunit is configured for collision avoidance in the event of recognisedrisk of collision.
 13. The numerically controlled laser machining toolaccording to claim 12, characterised in that the optical sensor systemcomprises at least two or preferably at least four cameras, preferablyCMOS cameras.
 14. The numerically controlled laser machining toolaccording to claim 13, characterised in that two cameras are provided,the capture areas of which are aligned in the same direction and thatthe capture area of a first camera captures a first half of themachining space and that the capture area of a second camera captures asecond half of the machining space.
 15. The numerically controlled lasermachining tool according to claim 13, characterised in that four camerasare provided and that the capture areas of two cameras capture a firsthalf of the machining space and that the capture areas of two camerascapture a second half of the machining space, wherein the two camerasare each arranged to be offset from one another.
 16. The numericallycontrolled laser machining tool according to claim 12, characterised inthat the optical sensor system is connected to the graphics processingunit with a high-speed connection.
 17. The numerically controlled lasermachining tool according to claim 12, characterised in that the graphicsprocessing unit is connected to the numerical control unit with areal-time Ethernet connection.
 18. The numerically controlled lasermachining tool according to claim 12, characterised in that the camerasystem and/or the graphics processing unit is configured to carry out afirst calibration of the intrinsic camera parameters and a secondcalibration of translation and rotation parameters of a coordinatesystem of the camera compared to a coordinate system of the lasermachining tool.